Wisconsin Diagnostic Breast Cancer (WDBC) Data: https://archive.ics.uci.edu/dataset/17/breast+cancer+wisconsin+diagnostic
# Import libraries necessary for this project
import numpy as np
import pandas as pd
from time import time
from IPython.display import display
import seaborn as sns
import matplotlib.pyplot as plt
import warnings
warnings.filterwarnings('ignore')
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier
from sklearn.tree import DecisionTreeClassifier
from sklearn.naive_bayes import GaussianNB
from sklearn.svm import SVC
from sklearn.model_selection import GridSearchCV, train_test_split
from sklearn.metrics import make_scorer, fbeta_score, accuracy_score, precision_score
%matplotlib inline
ls
Breast Cancer Wisconsin (Diagnostic) - UCI Machine Learning Repository.pdf
Task3.pdf
WDBC_ML_Project.ipynb
wdbc_data/
ls wdbc_data/
wdbc.data* wdbc.names*
cat wdbc_data/wdbc.names
1. Title: Wisconsin Diagnostic Breast Cancer (WDBC)
2. Source Information
a) Creators:
Dr. William H. Wolberg, General Surgery Dept., University of
Wisconsin, Clinical Sciences Center, Madison, WI 53792
wolberg@eagle.surgery.wisc.edu
W. Nick Street, Computer Sciences Dept., University of
Wisconsin, 1210 West Dayton St., Madison, WI 53706
street@cs.wisc.edu 608-262-6619
Olvi L. Mangasarian, Computer Sciences Dept., University of
Wisconsin, 1210 West Dayton St., Madison, WI 53706
olvi@cs.wisc.edu
b) Donor: Nick Street
c) Date: November 1995
3. Past Usage:
first usage:
W.N. Street, W.H. Wolberg and O.L. Mangasarian
Nuclear feature extraction for breast tumor diagnosis.
IS&T/SPIE 1993 International Symposium on Electronic Imaging: Science
and Technology, volume 1905, pages 861-870, San Jose, CA, 1993.
OR literature:
O.L. Mangasarian, W.N. Street and W.H. Wolberg.
Breast cancer diagnosis and prognosis via linear programming.
Operations Research, 43(4), pages 570-577, July-August 1995.
Medical literature:
W.H. Wolberg, W.N. Street, and O.L. Mangasarian.
Machine learning techniques to diagnose breast cancer from
fine-needle aspirates.
Cancer Letters 77 (1994) 163-171.
W.H. Wolberg, W.N. Street, and O.L. Mangasarian.
Image analysis and machine learning applied to breast cancer
diagnosis and prognosis.
Analytical and Quantitative Cytology and Histology, Vol. 17
No. 2, pages 77-87, April 1995.
W.H. Wolberg, W.N. Street, D.M. Heisey, and O.L. Mangasarian.
Computerized breast cancer diagnosis and prognosis from fine
needle aspirates.
Archives of Surgery 1995;130:511-516.
W.H. Wolberg, W.N. Street, D.M. Heisey, and O.L. Mangasarian.
Computer-derived nuclear features distinguish malignant from
benign breast cytology.
Human Pathology, 26:792--796, 1995.
See also:
http://www.cs.wisc.edu/~olvi/uwmp/mpml.html
http://www.cs.wisc.edu/~olvi/uwmp/cancer.html
Results:
- predicting field 2, diagnosis: B = benign, M = malignant
- sets are linearly separable using all 30 input features
- best predictive accuracy obtained using one separating plane
in the 3-D space of Worst Area, Worst Smoothness and
Mean Texture. Estimated accuracy 97.5% using repeated
10-fold crossvalidations. Classifier has correctly
diagnosed 176 consecutive new patients as of November
1995.
4. Relevant information
Features are computed from a digitized image of a fine needle
aspirate (FNA) of a breast mass. They describe
characteristics of the cell nuclei present in the image.
A few of the images can be found at
http://www.cs.wisc.edu/~street/images/
Separating plane described above was obtained using
Multisurface Method-Tree (MSM-T) [K. P. Bennett, "Decision Tree
Construction Via Linear Programming." Proceedings of the 4th
Midwest Artificial Intelligence and Cognitive Science Society,
pp. 97-101, 1992], a classification method which uses linear
programming to construct a decision tree. Relevant features
were selected using an exhaustive search in the space of 1-4
features and 1-3 separating planes.
The actual linear program used to obtain the separating plane
in the 3-dimensional space is that described in:
[K. P. Bennett and O. L. Mangasarian: "Robust Linear
Programming Discrimination of Two Linearly Inseparable Sets",
Optimization Methods and Software 1, 1992, 23-34].
This database is also available through the UW CS ftp server:
ftp ftp.cs.wisc.edu
cd math-prog/cpo-dataset/machine-learn/WDBC/
5. Number of instances: 569
6. Number of attributes: 32 (ID, diagnosis, 30 real-valued input features)
7. Attribute information
1) ID number
2) Diagnosis (M = malignant, B = benign)
3-32)
Ten real-valued features are computed for each cell nucleus:
a) radius (mean of distances from center to points on the perimeter)
b) texture (standard deviation of gray-scale values)
c) perimeter
d) area
e) smoothness (local variation in radius lengths)
f) compactness (perimeter^2 / area - 1.0)
g) concavity (severity of concave portions of the contour)
h) concave points (number of concave portions of the contour)
i) symmetry
j) fractal dimension ("coastline approximation" - 1)
Several of the papers listed above contain detailed descriptions of
how these features are computed.
The mean, standard error, and "worst" or largest (mean of the three
largest values) of these features were computed for each image,
resulting in 30 features. For instance, field 3 is Mean Radius, field
13 is Radius SE, field 23 is Worst Radius.
All feature values are recoded with four significant digits.
8. Missing attribute values: none
9. Class distribution: 357 benign, 212 malignant
Attribute information
1) ID number 2) Diagnosis (M = malignant, B = benign) 3-32)
Ten real-valued features are computed for each cell nucleus:
a) radius (mean of distances from center to points on the perimeter)
b) texture (standard deviation of gray-scale values)
c) perimeter
d) area
e) smoothness (local variation in radius lengths)
f) compactness (perimeter^2 / area - 1.0)
g) concavity (severity of concave portions of the contour)
h) concave points (number of concave portions of the contour)
i) symmetry
j) fractal dimension ("coastline approximation" - 1)
Several of the papers listed above contain detailed descriptions of how these features are computed.
The mean, standard error, and "worst" or largest (mean of the three largest values) of these features were computed for each image, resulting in 30 features. For instance, field 3 is Mean Radius, field 13 is Radius SE, field 23 is Worst Radius.
All feature values are recoded with four significant digits.
columns_name = ["ID","Diagnosis","radius-M","texture-M","perimeter-M","area-M",
"smoothness-M","compactness-M","concavity-M","concave_points-M","symmetry-M","fractal_dimension-M",
"radius-SE","texture-SE","perimeter-SE","area-SE",
"smoothness-SE","compactness-SE","concavity-SE","concave_points-SE","symmetry-SE","fractal_dimension-SE",
"radius-W","texture-W","perimeter-W","area-W",
"smoothness-W","compactness-W","concavity-W","concave_points-W","symmetry-W","fractal_dimension-W",]
wdbc_df = pd.read_table('wdbc_data/wdbc.data',
sep=',', names = columns_name)
wdbc_df.info()
<class 'pandas.core.frame.DataFrame'> RangeIndex: 569 entries, 0 to 568 Data columns (total 32 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 ID 569 non-null int64 1 Diagnosis 569 non-null object 2 radius-M 569 non-null float64 3 texture-M 569 non-null float64 4 perimeter-M 569 non-null float64 5 area-M 569 non-null float64 6 smoothness-M 569 non-null float64 7 compactness-M 569 non-null float64 8 concavity-M 569 non-null float64 9 concave_points-M 569 non-null float64 10 symmetry-M 569 non-null float64 11 fractal_dimension-M 569 non-null float64 12 radius-SE 569 non-null float64 13 texture-SE 569 non-null float64 14 perimeter-SE 569 non-null float64 15 area-SE 569 non-null float64 16 smoothness-SE 569 non-null float64 17 compactness-SE 569 non-null float64 18 concavity-SE 569 non-null float64 19 concave_points-SE 569 non-null float64 20 symmetry-SE 569 non-null float64 21 fractal_dimension-SE 569 non-null float64 22 radius-W 569 non-null float64 23 texture-W 569 non-null float64 24 perimeter-W 569 non-null float64 25 area-W 569 non-null float64 26 smoothness-W 569 non-null float64 27 compactness-W 569 non-null float64 28 concavity-W 569 non-null float64 29 concave_points-W 569 non-null float64 30 symmetry-W 569 non-null float64 31 fractal_dimension-W 569 non-null float64 dtypes: float64(30), int64(1), object(1) memory usage: 142.4+ KB
wdbc_df.head()
| ID | Diagnosis | radius-M | texture-M | perimeter-M | area-M | smoothness-M | compactness-M | concavity-M | concave_points-M | ... | radius-W | texture-W | perimeter-W | area-W | smoothness-W | compactness-W | concavity-W | concave_points-W | symmetry-W | fractal_dimension-W | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 842302 | M | 17.99 | 10.38 | 122.80 | 1001.0 | 0.11840 | 0.27760 | 0.3001 | 0.14710 | ... | 25.38 | 17.33 | 184.60 | 2019.0 | 0.1622 | 0.6656 | 0.7119 | 0.2654 | 0.4601 | 0.11890 |
| 1 | 842517 | M | 20.57 | 17.77 | 132.90 | 1326.0 | 0.08474 | 0.07864 | 0.0869 | 0.07017 | ... | 24.99 | 23.41 | 158.80 | 1956.0 | 0.1238 | 0.1866 | 0.2416 | 0.1860 | 0.2750 | 0.08902 |
| 2 | 84300903 | M | 19.69 | 21.25 | 130.00 | 1203.0 | 0.10960 | 0.15990 | 0.1974 | 0.12790 | ... | 23.57 | 25.53 | 152.50 | 1709.0 | 0.1444 | 0.4245 | 0.4504 | 0.2430 | 0.3613 | 0.08758 |
| 3 | 84348301 | M | 11.42 | 20.38 | 77.58 | 386.1 | 0.14250 | 0.28390 | 0.2414 | 0.10520 | ... | 14.91 | 26.50 | 98.87 | 567.7 | 0.2098 | 0.8663 | 0.6869 | 0.2575 | 0.6638 | 0.17300 |
| 4 | 84358402 | M | 20.29 | 14.34 | 135.10 | 1297.0 | 0.10030 | 0.13280 | 0.1980 | 0.10430 | ... | 22.54 | 16.67 | 152.20 | 1575.0 | 0.1374 | 0.2050 | 0.4000 | 0.1625 | 0.2364 | 0.07678 |
5 rows × 32 columns
wdbc_df.describe().T
| count | mean | std | min | 25% | 50% | 75% | max | |
|---|---|---|---|---|---|---|---|---|
| ID | 569.0 | 3.037183e+07 | 1.250206e+08 | 8670.000000 | 869218.000000 | 906024.000000 | 8.813129e+06 | 9.113205e+08 |
| radius-M | 569.0 | 1.412729e+01 | 3.524049e+00 | 6.981000 | 11.700000 | 13.370000 | 1.578000e+01 | 2.811000e+01 |
| texture-M | 569.0 | 1.928965e+01 | 4.301036e+00 | 9.710000 | 16.170000 | 18.840000 | 2.180000e+01 | 3.928000e+01 |
| perimeter-M | 569.0 | 9.196903e+01 | 2.429898e+01 | 43.790000 | 75.170000 | 86.240000 | 1.041000e+02 | 1.885000e+02 |
| area-M | 569.0 | 6.548891e+02 | 3.519141e+02 | 143.500000 | 420.300000 | 551.100000 | 7.827000e+02 | 2.501000e+03 |
| smoothness-M | 569.0 | 9.636028e-02 | 1.406413e-02 | 0.052630 | 0.086370 | 0.095870 | 1.053000e-01 | 1.634000e-01 |
| compactness-M | 569.0 | 1.043410e-01 | 5.281276e-02 | 0.019380 | 0.064920 | 0.092630 | 1.304000e-01 | 3.454000e-01 |
| concavity-M | 569.0 | 8.879932e-02 | 7.971981e-02 | 0.000000 | 0.029560 | 0.061540 | 1.307000e-01 | 4.268000e-01 |
| concave_points-M | 569.0 | 4.891915e-02 | 3.880284e-02 | 0.000000 | 0.020310 | 0.033500 | 7.400000e-02 | 2.012000e-01 |
| symmetry-M | 569.0 | 1.811619e-01 | 2.741428e-02 | 0.106000 | 0.161900 | 0.179200 | 1.957000e-01 | 3.040000e-01 |
| fractal_dimension-M | 569.0 | 6.279761e-02 | 7.060363e-03 | 0.049960 | 0.057700 | 0.061540 | 6.612000e-02 | 9.744000e-02 |
| radius-SE | 569.0 | 4.051721e-01 | 2.773127e-01 | 0.111500 | 0.232400 | 0.324200 | 4.789000e-01 | 2.873000e+00 |
| texture-SE | 569.0 | 1.216853e+00 | 5.516484e-01 | 0.360200 | 0.833900 | 1.108000 | 1.474000e+00 | 4.885000e+00 |
| perimeter-SE | 569.0 | 2.866059e+00 | 2.021855e+00 | 0.757000 | 1.606000 | 2.287000 | 3.357000e+00 | 2.198000e+01 |
| area-SE | 569.0 | 4.033708e+01 | 4.549101e+01 | 6.802000 | 17.850000 | 24.530000 | 4.519000e+01 | 5.422000e+02 |
| smoothness-SE | 569.0 | 7.040979e-03 | 3.002518e-03 | 0.001713 | 0.005169 | 0.006380 | 8.146000e-03 | 3.113000e-02 |
| compactness-SE | 569.0 | 2.547814e-02 | 1.790818e-02 | 0.002252 | 0.013080 | 0.020450 | 3.245000e-02 | 1.354000e-01 |
| concavity-SE | 569.0 | 3.189372e-02 | 3.018606e-02 | 0.000000 | 0.015090 | 0.025890 | 4.205000e-02 | 3.960000e-01 |
| concave_points-SE | 569.0 | 1.179614e-02 | 6.170285e-03 | 0.000000 | 0.007638 | 0.010930 | 1.471000e-02 | 5.279000e-02 |
| symmetry-SE | 569.0 | 2.054230e-02 | 8.266372e-03 | 0.007882 | 0.015160 | 0.018730 | 2.348000e-02 | 7.895000e-02 |
| fractal_dimension-SE | 569.0 | 3.794904e-03 | 2.646071e-03 | 0.000895 | 0.002248 | 0.003187 | 4.558000e-03 | 2.984000e-02 |
| radius-W | 569.0 | 1.626919e+01 | 4.833242e+00 | 7.930000 | 13.010000 | 14.970000 | 1.879000e+01 | 3.604000e+01 |
| texture-W | 569.0 | 2.567722e+01 | 6.146258e+00 | 12.020000 | 21.080000 | 25.410000 | 2.972000e+01 | 4.954000e+01 |
| perimeter-W | 569.0 | 1.072612e+02 | 3.360254e+01 | 50.410000 | 84.110000 | 97.660000 | 1.254000e+02 | 2.512000e+02 |
| area-W | 569.0 | 8.805831e+02 | 5.693570e+02 | 185.200000 | 515.300000 | 686.500000 | 1.084000e+03 | 4.254000e+03 |
| smoothness-W | 569.0 | 1.323686e-01 | 2.283243e-02 | 0.071170 | 0.116600 | 0.131300 | 1.460000e-01 | 2.226000e-01 |
| compactness-W | 569.0 | 2.542650e-01 | 1.573365e-01 | 0.027290 | 0.147200 | 0.211900 | 3.391000e-01 | 1.058000e+00 |
| concavity-W | 569.0 | 2.721885e-01 | 2.086243e-01 | 0.000000 | 0.114500 | 0.226700 | 3.829000e-01 | 1.252000e+00 |
| concave_points-W | 569.0 | 1.146062e-01 | 6.573234e-02 | 0.000000 | 0.064930 | 0.099930 | 1.614000e-01 | 2.910000e-01 |
| symmetry-W | 569.0 | 2.900756e-01 | 6.186747e-02 | 0.156500 | 0.250400 | 0.282200 | 3.179000e-01 | 6.638000e-01 |
| fractal_dimension-W | 569.0 | 8.394582e-02 | 1.806127e-02 | 0.055040 | 0.071460 | 0.080040 | 9.208000e-02 | 2.075000e-01 |
wdbc_df.isnull().values.any()
False
wdbc_df.shape
(569, 32)
malignant = (wdbc_df["Diagnosis"] == "M").sum()
malignant
212
benign = (wdbc_df["Diagnosis"] == "B").sum()
benign
357
labels = ['malignant','benign']
sizes = [malignant,benign]
#fig,ax = plt.subplots(figsize=(6,3))
plt.pie(sizes, labels=labels, autopct='%1.1f%%')
plt.title('Percentage of malignant and benign tumors.')
plt.show()
wdbc_df = wdbc_df.drop('ID', axis = 1)
wdbc_df['Diagnosis'] = wdbc_df['Diagnosis'].map({'B':0, 'M':1})
# Now, the 'diagnosis' column contains 0 for benign and 1 for malignant
# Calculate the correlation matrix
correlation_matrix = wdbc_df.corr()
correlation_matrix
| Diagnosis | radius-M | texture-M | perimeter-M | area-M | smoothness-M | compactness-M | concavity-M | concave_points-M | symmetry-M | ... | radius-W | texture-W | perimeter-W | area-W | smoothness-W | compactness-W | concavity-W | concave_points-W | symmetry-W | fractal_dimension-W | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Diagnosis | 1.000000 | 0.730029 | 0.415185 | 0.742636 | 0.708984 | 0.358560 | 0.596534 | 0.696360 | 0.776614 | 0.330499 | ... | 0.776454 | 0.456903 | 0.782914 | 0.733825 | 0.421465 | 0.590998 | 0.659610 | 0.793566 | 0.416294 | 0.323872 |
| radius-M | 0.730029 | 1.000000 | 0.323782 | 0.997855 | 0.987357 | 0.170581 | 0.506124 | 0.676764 | 0.822529 | 0.147741 | ... | 0.969539 | 0.297008 | 0.965137 | 0.941082 | 0.119616 | 0.413463 | 0.526911 | 0.744214 | 0.163953 | 0.007066 |
| texture-M | 0.415185 | 0.323782 | 1.000000 | 0.329533 | 0.321086 | -0.023389 | 0.236702 | 0.302418 | 0.293464 | 0.071401 | ... | 0.352573 | 0.912045 | 0.358040 | 0.343546 | 0.077503 | 0.277830 | 0.301025 | 0.295316 | 0.105008 | 0.119205 |
| perimeter-M | 0.742636 | 0.997855 | 0.329533 | 1.000000 | 0.986507 | 0.207278 | 0.556936 | 0.716136 | 0.850977 | 0.183027 | ... | 0.969476 | 0.303038 | 0.970387 | 0.941550 | 0.150549 | 0.455774 | 0.563879 | 0.771241 | 0.189115 | 0.051019 |
| area-M | 0.708984 | 0.987357 | 0.321086 | 0.986507 | 1.000000 | 0.177028 | 0.498502 | 0.685983 | 0.823269 | 0.151293 | ... | 0.962746 | 0.287489 | 0.959120 | 0.959213 | 0.123523 | 0.390410 | 0.512606 | 0.722017 | 0.143570 | 0.003738 |
| smoothness-M | 0.358560 | 0.170581 | -0.023389 | 0.207278 | 0.177028 | 1.000000 | 0.659123 | 0.521984 | 0.553695 | 0.557775 | ... | 0.213120 | 0.036072 | 0.238853 | 0.206718 | 0.805324 | 0.472468 | 0.434926 | 0.503053 | 0.394309 | 0.499316 |
| compactness-M | 0.596534 | 0.506124 | 0.236702 | 0.556936 | 0.498502 | 0.659123 | 1.000000 | 0.883121 | 0.831135 | 0.602641 | ... | 0.535315 | 0.248133 | 0.590210 | 0.509604 | 0.565541 | 0.865809 | 0.816275 | 0.815573 | 0.510223 | 0.687382 |
| concavity-M | 0.696360 | 0.676764 | 0.302418 | 0.716136 | 0.685983 | 0.521984 | 0.883121 | 1.000000 | 0.921391 | 0.500667 | ... | 0.688236 | 0.299879 | 0.729565 | 0.675987 | 0.448822 | 0.754968 | 0.884103 | 0.861323 | 0.409464 | 0.514930 |
| concave_points-M | 0.776614 | 0.822529 | 0.293464 | 0.850977 | 0.823269 | 0.553695 | 0.831135 | 0.921391 | 1.000000 | 0.462497 | ... | 0.830318 | 0.292752 | 0.855923 | 0.809630 | 0.452753 | 0.667454 | 0.752399 | 0.910155 | 0.375744 | 0.368661 |
| symmetry-M | 0.330499 | 0.147741 | 0.071401 | 0.183027 | 0.151293 | 0.557775 | 0.602641 | 0.500667 | 0.462497 | 1.000000 | ... | 0.185728 | 0.090651 | 0.219169 | 0.177193 | 0.426675 | 0.473200 | 0.433721 | 0.430297 | 0.699826 | 0.438413 |
| fractal_dimension-M | -0.012838 | -0.311631 | -0.076437 | -0.261477 | -0.283110 | 0.584792 | 0.565369 | 0.336783 | 0.166917 | 0.479921 | ... | -0.253691 | -0.051269 | -0.205151 | -0.231854 | 0.504942 | 0.458798 | 0.346234 | 0.175325 | 0.334019 | 0.767297 |
| radius-SE | 0.567134 | 0.679090 | 0.275869 | 0.691765 | 0.732562 | 0.301467 | 0.497473 | 0.631925 | 0.698050 | 0.303379 | ... | 0.715065 | 0.194799 | 0.719684 | 0.751548 | 0.141919 | 0.287103 | 0.380585 | 0.531062 | 0.094543 | 0.049559 |
| texture-SE | -0.008303 | -0.097317 | 0.386358 | -0.086761 | -0.066280 | 0.068406 | 0.046205 | 0.076218 | 0.021480 | 0.128053 | ... | -0.111690 | 0.409003 | -0.102242 | -0.083195 | -0.073658 | -0.092439 | -0.068956 | -0.119638 | -0.128215 | -0.045655 |
| perimeter-SE | 0.556141 | 0.674172 | 0.281673 | 0.693135 | 0.726628 | 0.296092 | 0.548905 | 0.660391 | 0.710650 | 0.313893 | ... | 0.697201 | 0.200371 | 0.721031 | 0.730713 | 0.130054 | 0.341919 | 0.418899 | 0.554897 | 0.109930 | 0.085433 |
| area-SE | 0.548236 | 0.735864 | 0.259845 | 0.744983 | 0.800086 | 0.246552 | 0.455653 | 0.617427 | 0.690299 | 0.223970 | ... | 0.757373 | 0.196497 | 0.761213 | 0.811408 | 0.125389 | 0.283257 | 0.385100 | 0.538166 | 0.074126 | 0.017539 |
| smoothness-SE | -0.067016 | -0.222600 | 0.006614 | -0.202694 | -0.166777 | 0.332375 | 0.135299 | 0.098564 | 0.027653 | 0.187321 | ... | -0.230691 | -0.074743 | -0.217304 | -0.182195 | 0.314457 | -0.055558 | -0.058298 | -0.102007 | -0.107342 | 0.101480 |
| compactness-SE | 0.292999 | 0.206000 | 0.191975 | 0.250744 | 0.212583 | 0.318943 | 0.738722 | 0.670279 | 0.490424 | 0.421659 | ... | 0.204607 | 0.143003 | 0.260516 | 0.199371 | 0.227394 | 0.678780 | 0.639147 | 0.483208 | 0.277878 | 0.590973 |
| concavity-SE | 0.253730 | 0.194204 | 0.143293 | 0.228082 | 0.207660 | 0.248396 | 0.570517 | 0.691270 | 0.439167 | 0.342627 | ... | 0.186904 | 0.100241 | 0.226680 | 0.188353 | 0.168481 | 0.484858 | 0.662564 | 0.440472 | 0.197788 | 0.439329 |
| concave_points-SE | 0.408042 | 0.376169 | 0.163851 | 0.407217 | 0.372320 | 0.380676 | 0.642262 | 0.683260 | 0.615634 | 0.393298 | ... | 0.358127 | 0.086741 | 0.394999 | 0.342271 | 0.215351 | 0.452888 | 0.549592 | 0.602450 | 0.143116 | 0.310655 |
| symmetry-SE | -0.006522 | -0.104321 | 0.009127 | -0.081629 | -0.072497 | 0.200774 | 0.229977 | 0.178009 | 0.095351 | 0.449137 | ... | -0.128121 | -0.077473 | -0.103753 | -0.110343 | -0.012662 | 0.060255 | 0.037119 | -0.030413 | 0.389402 | 0.078079 |
| fractal_dimension-SE | 0.077972 | -0.042641 | 0.054458 | -0.005523 | -0.019887 | 0.283607 | 0.507318 | 0.449301 | 0.257584 | 0.331786 | ... | -0.037488 | -0.003195 | -0.001000 | -0.022736 | 0.170568 | 0.390159 | 0.379975 | 0.215204 | 0.111094 | 0.591328 |
| radius-W | 0.776454 | 0.969539 | 0.352573 | 0.969476 | 0.962746 | 0.213120 | 0.535315 | 0.688236 | 0.830318 | 0.185728 | ... | 1.000000 | 0.359921 | 0.993708 | 0.984015 | 0.216574 | 0.475820 | 0.573975 | 0.787424 | 0.243529 | 0.093492 |
| texture-W | 0.456903 | 0.297008 | 0.912045 | 0.303038 | 0.287489 | 0.036072 | 0.248133 | 0.299879 | 0.292752 | 0.090651 | ... | 0.359921 | 1.000000 | 0.365098 | 0.345842 | 0.225429 | 0.360832 | 0.368366 | 0.359755 | 0.233027 | 0.219122 |
| perimeter-W | 0.782914 | 0.965137 | 0.358040 | 0.970387 | 0.959120 | 0.238853 | 0.590210 | 0.729565 | 0.855923 | 0.219169 | ... | 0.993708 | 0.365098 | 1.000000 | 0.977578 | 0.236775 | 0.529408 | 0.618344 | 0.816322 | 0.269493 | 0.138957 |
| area-W | 0.733825 | 0.941082 | 0.343546 | 0.941550 | 0.959213 | 0.206718 | 0.509604 | 0.675987 | 0.809630 | 0.177193 | ... | 0.984015 | 0.345842 | 0.977578 | 1.000000 | 0.209145 | 0.438296 | 0.543331 | 0.747419 | 0.209146 | 0.079647 |
| smoothness-W | 0.421465 | 0.119616 | 0.077503 | 0.150549 | 0.123523 | 0.805324 | 0.565541 | 0.448822 | 0.452753 | 0.426675 | ... | 0.216574 | 0.225429 | 0.236775 | 0.209145 | 1.000000 | 0.568187 | 0.518523 | 0.547691 | 0.493838 | 0.617624 |
| compactness-W | 0.590998 | 0.413463 | 0.277830 | 0.455774 | 0.390410 | 0.472468 | 0.865809 | 0.754968 | 0.667454 | 0.473200 | ... | 0.475820 | 0.360832 | 0.529408 | 0.438296 | 0.568187 | 1.000000 | 0.892261 | 0.801080 | 0.614441 | 0.810455 |
| concavity-W | 0.659610 | 0.526911 | 0.301025 | 0.563879 | 0.512606 | 0.434926 | 0.816275 | 0.884103 | 0.752399 | 0.433721 | ... | 0.573975 | 0.368366 | 0.618344 | 0.543331 | 0.518523 | 0.892261 | 1.000000 | 0.855434 | 0.532520 | 0.686511 |
| concave_points-W | 0.793566 | 0.744214 | 0.295316 | 0.771241 | 0.722017 | 0.503053 | 0.815573 | 0.861323 | 0.910155 | 0.430297 | ... | 0.787424 | 0.359755 | 0.816322 | 0.747419 | 0.547691 | 0.801080 | 0.855434 | 1.000000 | 0.502528 | 0.511114 |
| symmetry-W | 0.416294 | 0.163953 | 0.105008 | 0.189115 | 0.143570 | 0.394309 | 0.510223 | 0.409464 | 0.375744 | 0.699826 | ... | 0.243529 | 0.233027 | 0.269493 | 0.209146 | 0.493838 | 0.614441 | 0.532520 | 0.502528 | 1.000000 | 0.537848 |
| fractal_dimension-W | 0.323872 | 0.007066 | 0.119205 | 0.051019 | 0.003738 | 0.499316 | 0.687382 | 0.514930 | 0.368661 | 0.438413 | ... | 0.093492 | 0.219122 | 0.138957 | 0.079647 | 0.617624 | 0.810455 | 0.686511 | 0.511114 | 0.537848 | 1.000000 |
31 rows × 31 columns
# Create the heatmap
plt.figure(figsize=(32,24))
sns.set(font_scale=1.5) # Adjust the font scale if needed
sns.heatmap(correlation_matrix, annot=True, cmap="coolwarm", linewidths=.5)
plt.title('Correlation Heatmap')
# Adjust the figure size if needed
plt.show()
wdbc_df.info()
<class 'pandas.core.frame.DataFrame'> RangeIndex: 569 entries, 0 to 568 Data columns (total 31 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 Diagnosis 569 non-null int64 1 radius-M 569 non-null float64 2 texture-M 569 non-null float64 3 perimeter-M 569 non-null float64 4 area-M 569 non-null float64 5 smoothness-M 569 non-null float64 6 compactness-M 569 non-null float64 7 concavity-M 569 non-null float64 8 concave_points-M 569 non-null float64 9 symmetry-M 569 non-null float64 10 fractal_dimension-M 569 non-null float64 11 radius-SE 569 non-null float64 12 texture-SE 569 non-null float64 13 perimeter-SE 569 non-null float64 14 area-SE 569 non-null float64 15 smoothness-SE 569 non-null float64 16 compactness-SE 569 non-null float64 17 concavity-SE 569 non-null float64 18 concave_points-SE 569 non-null float64 19 symmetry-SE 569 non-null float64 20 fractal_dimension-SE 569 non-null float64 21 radius-W 569 non-null float64 22 texture-W 569 non-null float64 23 perimeter-W 569 non-null float64 24 area-W 569 non-null float64 25 smoothness-W 569 non-null float64 26 compactness-W 569 non-null float64 27 concavity-W 569 non-null float64 28 concave_points-W 569 non-null float64 29 symmetry-W 569 non-null float64 30 fractal_dimension-W 569 non-null float64 dtypes: float64(30), int64(1) memory usage: 137.9 KB
result = wdbc_df['Diagnosis']
features = wdbc_df.drop('Diagnosis',axis = 1)
result.head()
0 1 1 1 2 1 3 1 4 1 Name: Diagnosis, dtype: int64
features.head()
| radius-M | texture-M | perimeter-M | area-M | smoothness-M | compactness-M | concavity-M | concave_points-M | symmetry-M | fractal_dimension-M | ... | radius-W | texture-W | perimeter-W | area-W | smoothness-W | compactness-W | concavity-W | concave_points-W | symmetry-W | fractal_dimension-W | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 17.99 | 10.38 | 122.80 | 1001.0 | 0.11840 | 0.27760 | 0.3001 | 0.14710 | 0.2419 | 0.07871 | ... | 25.38 | 17.33 | 184.60 | 2019.0 | 0.1622 | 0.6656 | 0.7119 | 0.2654 | 0.4601 | 0.11890 |
| 1 | 20.57 | 17.77 | 132.90 | 1326.0 | 0.08474 | 0.07864 | 0.0869 | 0.07017 | 0.1812 | 0.05667 | ... | 24.99 | 23.41 | 158.80 | 1956.0 | 0.1238 | 0.1866 | 0.2416 | 0.1860 | 0.2750 | 0.08902 |
| 2 | 19.69 | 21.25 | 130.00 | 1203.0 | 0.10960 | 0.15990 | 0.1974 | 0.12790 | 0.2069 | 0.05999 | ... | 23.57 | 25.53 | 152.50 | 1709.0 | 0.1444 | 0.4245 | 0.4504 | 0.2430 | 0.3613 | 0.08758 |
| 3 | 11.42 | 20.38 | 77.58 | 386.1 | 0.14250 | 0.28390 | 0.2414 | 0.10520 | 0.2597 | 0.09744 | ... | 14.91 | 26.50 | 98.87 | 567.7 | 0.2098 | 0.8663 | 0.6869 | 0.2575 | 0.6638 | 0.17300 |
| 4 | 20.29 | 14.34 | 135.10 | 1297.0 | 0.10030 | 0.13280 | 0.1980 | 0.10430 | 0.1809 | 0.05883 | ... | 22.54 | 16.67 | 152.20 | 1575.0 | 0.1374 | 0.2050 | 0.4000 | 0.1625 | 0.2364 | 0.07678 |
5 rows × 30 columns
def train_predict(model, X_train, y_train, X_val, y_val):
'''
inputs:
- learner: the learning algorithm to be trained and predicted on
- sample_size: the size of samples (number) to be drawn from training set
- X_train: features training set
- y_train: income training set
- X_test: features testing set
- y_test: income testing set
'''
result = {}
time_consum = {}
# TODO: Fit the learner to the training data using slicing with 'sample_size' using .fit(training_features[:], training_labels[:])
start = time() # Get start time
model.fit(X_train, y_train)
end = time() # Get end time
# TODO: Calculate the training time
time_consum['train_time'] = end - start
# TODO: Get the predictions on the test set(X_test),
# then get predictions on the first 300 training samples(X_train) using .predict()
start = time() # Get start time
y_pred = model.predict(X_val)
predictions_train = model.predict(X_train)
end = time() # Get end time
# TODO: Calculate the total prediction time
time_consum['pred_time'] = end - start
# TODO: Compute accuracy on test set
result['acc_test'] = accuracy_score(y_val,y_pred)
result['pec_test'] = precision_score(y_val,y_pred)
# TODO: Compute F-score on testing set which is y_test
result['f_test'] = fbeta_score(y_val, y_pred, beta=0.5)
# Success
#print("{} model output.".format(model.__class__.__name__))
# Return the results
return result,time_consum
def evaluate(typeE, dictInfo):
tyep_graph = typeE
if tyep_graph == 1:
item_names = list(time_consum.keys())
item_values_train = [data['train_time'] for data in time_consum.values()]
item_values_pred = [data['pred_time'] for data in time_consum.values()]
# Create a bar chart
plt.figure(figsize=(20, 15)) # Optional: Adjust figure size
bars_train = plt.bar(item_names, item_values_train, label='Training Time', width=0.5, align='center')
bars_pred = plt.bar(item_names, item_values_pred, label='Prediction Time', width=0.5, align='edge')
# Add values on top of the training time bars
for bar, value in zip(bars_train, item_values_train):
plt.text(bar.get_x() + bar.get_width() / 2, value, str(round(value, 4)), ha='center', va='bottom',rotation=45)
# Add values on top of the prediction time bars
for bar, value in zip(bars_pred, item_values_pred):
plt.text(bar.get_x() + bar.get_width() / 2, value, str(round(value, 4)), ha='center', va='bottom',rotation=45)
# Add labels and title
plt.xlabel('Machine learninng Models', fontsize=24)
plt.ylabel('Time consuming(seconds)', fontsize=24)
plt.title('Training and Prediction Times for Different Models',fontsize=24)
plt.legend(loc='lower center', bbox_to_anchor=(1, 1))
# Show the chart
plt.xticks(rotation=45) # Optional: Rotate x-axis labels for better readability
plt.tight_layout() # Optional: Adjust layout for better spacing
plt.show()
else:
item_names = list(results.keys())
item_acc_score = [data['acc_test'] for data in results.values()]
item_precision = [data['pec_test'] for data in results.values()]
item_f_score = [data['f_test'] for data in results.values()]
plt.figure(figsize=(20, 15))
# Define the positions for each group of bars
x = np.arange(len(item_names))
# Set the width of each group of bars
bar_width = 0.25
# Create a bar chart
bars_acc = plt.bar(x - bar_width, item_acc_score, label='Accuracy Score', width=bar_width, align='edge')
bars_pec = plt.bar(x, item_precision, label='Precision Score', width=bar_width, align='edge')
bars_fs = plt.bar(x + bar_width, item_f_score, label='F-score', width=bar_width, align='edge')
# Add values on top of the bars
for bars, values in [(bars_acc, item_acc_score), (bars_pec, item_precision), (bars_fs, item_f_score)]:
for bar, value in zip(bars, values):
plt.text(bar.get_x() + bar.get_width() / 2, value, str(round(value, 3)), ha='center', va='bottom', rotation=45)
# Customize x-axis labels
plt.xticks(x, item_names)
# Add labels and title
plt.xlabel('Machine learninng Models', fontsize=24)
plt.ylabel('Score', fontsize=24)
plt.title('Accuracy,Precision and Fbeta Score for Different Models', fontsize=24)
plt.legend(loc='lower center', bbox_to_anchor=(1, 1))
# Show the chart
plt.xticks(rotation=45) # Optional: Rotate x-axis labels for better readability
#plt.tight_layout() # Optional: Adjust layout for better spacing
plt.show()
X_train, X_temp, y_train, y_temp = train_test_split(features,
result,
test_size=0.3,
random_state=42)
X_val, X_test, y_val, y_test = train_test_split(X_temp,
y_temp,
test_size=0.5,
random_state=42)
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)
results = {}
time_consum = {}
# Create a Logistic Regression model
model1 = LogisticRegression()
model2 = DecisionTreeClassifier()
model3 = RandomForestClassifier()
model4 = GaussianNB()
model5 = SVC(kernel='linear')
# Train the model on the training data
models = [model1,model2,model3,model4,model5]
for i in range (5):
model_name = models[i].__class__.__name__
results[model_name] = {}
time_consum [model_name] = {}
results[model_name],time_consum[model_name] = train_predict(models[i], X_train_scaled, y_train, X_test_scaled, y_test)
evaluate(1, time_consum)
evaluate(2,results)
# def fine_turn(model,parameter):
# clf = model
# # TODO: Create the parameters list you wish to tune, using a dictionary if needed.
# # HINT: parameters = {'parameter_1': [value1, value2], 'parameter_2': [value1, value2]}
# parameters = parameter
# # TODO: Make an fbeta_score scoring object using make_scorer()
# beta = 0.5
# scorer = make_scorer(fbeta_score, beta=beta)
# # TODO: Perform grid search on the classifier using 'scorer' as the scoring method using GridSearchCV()
# grid_obj = GridSearchCV(clf,parameters, scoring=scorer)
# # TODO: Fit the grid search object to the training data and find the optimal parameters using fit()
# grid_fit = grid_obj.fit(X_train, y_train)
# # Get the estimator
# best_clf = grid_fit.best_estimator_
# # Make predictions using the unoptimized and model
# predictions = (clf.fit(X_train, y_train)).predict(X_val)
# best_predictions = best_clf.predict(X_val)
# # Report the before-and-afterscores
# print("Score report for {} ".format(clf.__class__.__name__))
# print("Unoptimized model\n------")
# print("Accuracy score on Validation data: {:.4f}".format(accuracy_score(y_val, predictions)))
# print("Precision score on Validation data: {:.4f}".format(precision_score(y_val, predictions)))
# print("F-score on Validation data: {:.4f}".format(fbeta_score(y_val, predictions, beta = 0.5)))
# print("\nOptimized Model\n------")
# print("Final accuracy score on the Validation data: {:.4f}".format(accuracy_score(y_val, best_predictions)))
# print("Final precision score on Validation data: {:.4f}".format(precision_score(y_val,best_predictions)))
# print("Final F-score on the Validation data: {:.4f}".format(fbeta_score(y_val, best_predictions, beta = 0.5)))
# clf1 = LogisticRegression()
# parameters1 = {'penalty': ['l1', 'l2'],
# 'C': [0.001, 0.01, 0.1, 1, 10],
# 'solver': ['liblinear'],
# 'max_iter': [100, 200, 300]}
# clf2 = RandomForestClassifier()
# parameters2 = {
# 'n_estimators': [100, 200, 300],
# 'max_depth': [None, 10, 20, 30],
# 'max_features': ['sqrt', 'log2', None],
# }
# fine_turn(clf1,parameters1)
# print('\n ============================================== \n')
# fine_turn(clf2,parameters2)
# def final_eval(model):
# y_pred_test = model.predict(X_test)
# Calculate test set metrics
# test_accuracy = accuracy_score(y_test, y_pred_test)
# test_precision = precision_score(y_test, y_pred_test)
# test_f1 = fbeta_score(y_test, y_pred_test, beta=0.5)
# Print test set metrics
# print("\nFinal Evaluation with {} on Test Set:".format(model.__class__.__name__))
# print(f"Test Accuracy: {test_accuracy:.4f}")
# print(f"Test Precision: {test_precision:.4f}")
# print(f"Test F1-Score: {test_f1:.4f}")
# for i in [clf1,clf2]:
# final_eval(i)
Choose a scikit-learn supervised learning algorithm that has a feature_importance_ attribute availble for it. This attribute is a function that ranks the importance of each feature when making predictions based on the chosen algorithm.
'.feature_importances_'.def feature_plot(importances, X_train):
# Display the five most important features
indices = np.argsort(importances)[::-1]
columns = X_train.columns.values[indices[:5]]
values = importances[indices][:5]
# Creat the plot
fig = plt.figure(figsize = (16,10))
plt.title("Normalized Weights for First Five Most Predictive Features", fontsize = 20)
plt.bar(np.arange(5), values, width = 0.4, align="center", color = '#00A000', \
label = "Feature Weight")
plt.bar(np.arange(5) - 0.3, np.cumsum(values), width = 0.2, align = "center", color = '#00A0A0', \
label = "Cumulative Feature Weight")
plt.xticks(np.arange(5), columns)
plt.xlim((-0.5, 4.5))
plt.ylim(0, 1)
plt.ylabel("Weight", fontsize = 20)
plt.xlabel("Feature", fontsize = 20)
plt.legend(loc = 'upper center')
plt.tight_layout()
plt.show()
#RandomForestClassifier after fine turn
# TODO: Extract the feature importances using .feature_importances_
importances2 = model3.feature_importances_
# Plot
feature_plot(importances2, X_train)
indices = np.argsort(importances2)[::-1]
columns = X_train.columns.values[indices[:5]]
values = importances2[indices][:5]
for i in range(len(columns)):
print("Weight for {} is: {:.4f}".format(columns[i], values[i]))
Weight for area-W is: 0.1225 Weight for perimeter-W is: 0.1157 Weight for concave_points-W is: 0.1096 Weight for concave_points-M is: 0.1021 Weight for radius-W is: 0.0808
# Sort the feature importances in descending order
sorted_indices = np.argsort(importances2)[::-1]
sorted_importances = importances2[sorted_indices]
# Calculate the cumulative feature weight
cumulative_importance = np.cumsum(sorted_importances)
# Print the top 5 cumulative feature weights
print("Top 5 Cumulative Feature Weights:")
for i in range(5):
feature_index = sorted_indices[i]
feature_weight = cumulative_importance[i]
print("Feature {}: {:.4f}".format(feature_index, feature_weight))
Top 5 Cumulative Feature Weights: Feature 23: 0.1225 Feature 22: 0.2382 Feature 27: 0.3478 Feature 7: 0.4500 Feature 20: 0.5308
plt.figure(figsize=(32,24))
sns.set(font_scale=1.5) # Adjust the font scale if needed
sns.heatmap(correlation_matrix, annot=True, cmap="coolwarm", linewidths=.5)
plt.title('Correlation Heatmap')
# Adjust the figure size if needed
plt.show()
feature_plot(importances2, X_train)